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work  was  funded  by  the  Quality  and  Productivity  Improvement 
Directorate  and  is  part  of  a  comprehensive  program  aimed  at 
implementing  the  principles  of  total  quality  management  in  the  execution 
of  the  Center's  mission. 
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ABSTRACT 


The  author’s  task,  assigned  by  the  Head  of  the  Quality  and 
Productivity  Improvement  Directorate,  was  to  find  a  method  for  choosing 
the  measures  of  effectiveness  for  the  Naval  Weapons  Center.  This  report 
describes  the  architecture  for  a  total  quality  management  measurement 
model  that  provides  a  vehicle  for  identifying  the  critical  measures  of 
effectiveness  for  NWC.  The  report  is  a  synthesis  of  information  gleaned 
from  the  literature  and  especially  from  consultation  with  Qualtec  Inc.,  West 
Palm  Beach  Gardens,  Florida,  the  consulting  arm  of  Florida  Power  and 
Light. 
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INTRODUCTION 


Many  of  the  key  elements  necessary  for  total  quality  management 
(TQM)  already  exist  at  the  Naval  Weapons  Center  (NWC).  For  instance,  NWC 
already  has  a  vision  statement,  a  philosophy  statement,  a  definitive 
mission  statement,  and  a  commitment  "to  excellence  through  teamwork 
and  innovation."  Many  processes  and  procedures  are  in  place,  and  a 
multitude  of  measurements  are  already  reported  both  internally  and 
externally.  Additionally,  some  formal  and  informal  surveys  of  customers 
and  emplo^tees  are  performed  on  and  off  Center  (NWC,  July  1990).  Many 
important  ingredients,  such  as  individual  commitment,  task  teams,  and 
especially  attention  to  customers  have  long  been  NWC  standards  (NWC, 
1968). 

This  report  describes  a  TQM  measurement  model  that  can  realign  the 
existing  elements,  processes,  procedures,  and  measurements  into  a  new 
way  of  using  the  available  information.  The  potential  benefits  are 
numerous.  Firstly,  the  model  provides  a  way  to  use  the  existing  system. 
Secondly,  the  model  easily  allows  the  definition  of  new  measures,  if 
necessary.  Thirdly,  the  model  provides  a  mechanism  through  which 
priorities  can  be  advertised  throughout  the  organization.  Fourthly,  the 
model  offers  a  means  by  which  the  accumulated  improvements  of  products 
by  employees  can  be  collectively  analyzed  to  show  response  to  the 
customers'  expectations. 

Most  importantly,  this  model  allows  the  definition  of  customer 
requirements  in  terms  of  the  technical  functions  performed  by  NWC 
employees,  and  the  prioritizing  of  these  functions  from  a  customer 
perspective.  The  best  possible  trade-off  of  all  requirements  can  be 
attained,  which  is  an  especially  important  factor  when  one  considers  that 
some  requirements  directly  conflict  with  one  another. 

This  measurement  model  can  create  a  framework  for  consistent 
language  and  communication  throughout  the  Center.  It  can  create  an  arena 
that  allows  efficient  flow-down  of  mission  and  policy  and  efficient  flow-up 
of  results.  That  is,  the  mission  of  the  Center  can  be  deployed  throughout 
the  Center.  Implementing  this  measurement  model  will  encourage,  the 
realization  of  TQM  at  N^C. 
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DESCRIPTION  OF  THE  TQM  MEASUREMENT  MODEL 


BASIC  DESIGN 

The  TQM  measurement  model  is  organized  by  broad  functional 
categories.  This  structure  minimizes  the  overlap  that  would  result  from 
categorizing  by  organizational  units  (Qualtec).  Many  functions  are  similar 
from  unit  to  unit.  The  elements  of  the  model  are  shown  in  the  foldout 
diagram  on  page  39  and  are  discussed  below. 

The  basic  design  of  the  model  is  the  same  at  all  levels.  The  whole 
organization  can  use  the  same  model  for  determining  priorities  from  a 
customer  perspective.  This  consistency  contributes  to  flow-down  of 
mission  and  policy  and  flow-up  of  improvement  effects;  that  is,  a  common 
measurement  system  contributes  to  consistent  communication  at  all  levels 
of  management  and  across  all  organizational  lines. 


PRODUCTS 

NWC  products  and  services  include  major  technical  programs,  test  and 
evaluation  activities,  manufacturing  technology  and  production  support, 
research,  development,  and  all  of  the  services  that  support  these  activities 
(NWC,  July  1990). 


CUSTOMERS 

Four  types  of  customers  must  be  considered.  Internal  customers  tend 
to  be  those  who  work  on  different  phases  of  a  product.  External  customers 
tend  to  be  those  off-Center  groups  that  use  an  end  product.  Direct 
customers  are  those  who  buy  and  use  a  product,  and  indirect  customers 
influence  how  business  is  done. 

NWC's  direct  external  customers  are  its  sponsors  in  Washington  that 
provide  the  funding  for  the  RDT&E  projects  (NWC,  July  1990).  Indirect 
customers  are  the  various  government  agencies  that  dictate  business 
practices;  the  regulatory  agencies  that  act  in  the  interest  of  the  public;  and, 
in  particular,  the  taxpayers. 
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CUSTOMER  REQUIREMENTS 


Customer  requirements  are  the  needs  and  wants  of  the  customer 
expressed  in  terms  of  specifications  mutually  agreed  upon  by  the  customer 
and  NWC.  Some  requirements  are  imposed.  Communication  with  customers 
should  produce  a  list  of  valid  requirements  that  are  current,  realistic,  and 
measurable;  that  meet  reasonable  expectations  of  the  customer;  and  that 
fall  within  corporate  constraints. 

The  customer  requirements  will  generally  address  areas  of  concern 
such  as  performance,  conformance,  durability,  affordability,  and  cost. 
Appendix  A  lists  several  possible  requirements  and  their  definitions. 

CATEGORIES 

All  customer  requirements  can  be  partitioned  into  the  following 
categories  (Figure  1): 

•  Quality  (customer  support;  direct  contact  with  customer) 

•  Deliverables  (basic  product  or  service  to  customer) 

•  Cost  (financial  management) 

•  Safety/Security  (protection  of  life  and  property) 

•  Corporate  Responsibility  (concern  for  community) 
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FIGURE  1.  Categories. 

The  functions  of  an  organization  (those  activities  and  processes  that,  in 
the  aggregate,  compose  the  daily  work  of  the  organization)  can  also  be 
partitioned  into  these  same  categories.  Thus  all  requirements  and  their 
corresponding  NWC  activities  are  categorized  in  the  same  way. 

Each  category  must  show  continuous  improvement.  With  limited 
resources  to  distribute  among  the  categories,  each  department  and 
organizational  level  must  have  an  understanding  of  priorities  from  the 
customer  perspective  in  order  to  maximize  resource  allocation. 
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NWC  ACTIVITIES 


NWC  activities  (Figure  2)  are  the  functions  within  NWC  that  best 
represent  the  customer  requirements.  Each  category  the  measurement 
model  has  several  NWC  activities. 


CATEC^IES  - 

QUALITY 

deliverables' 

"  SAFETY/  l; 
SECURITY  " 

CORPORATE 

RESPONSIBILITY  ^ 

. 

1  —  _ 





— 

— 

||  t  ■! 

l| 

JB 

>MER 

EMENT 

L 

>MER 
EMENT  ; 

CUSTC 
,  REQUIR 

=  1 

)MER 
EMENT  ; 

CUSTC 

,  REQUIR 
3 

)MER 

EMENT 

NWC 

,  ACnVITY 

m 

NWC 

ACnVITY 

NWC 

ACTIVITY 

iSit^ 

NWC 

ACTIVITY 

NWC 

,  Acnvrr 

FIGURE  2.  NWC  Activities. 

The  relationship  of  each  requirement  to  one  or  more  activities  must  be 
established.  That  is,  requirements  must  be  correlated  with  the  way 
business  is  done  at  NWC,  according  to  the  best  in-house  judgement.  Put 
another  way,  requirements  are  expressed  in  terms  of  operational 
definitions  (Kirkham).  These  defined  relationships  determine  the  NWC 
activities. 

For  example,  if  a  customer  requirement  is  ethical/legal  integrity,  then 
one  NWC  activity  could  be  addressing  equal  employment  opportunity 
issues  If  system  reliability  is  a  requirement,  then  some  NWC  activities 
could  be  program  management,  design  analysis,  parts  control,  development 
testing,  and  environmental  stress  screening. 

Each  activity  is  the  NWC  function  that  best  represents  a  customer 
requirement.  A  Quality  Table  (Table  1),  described  later  in  this  report, 
offers  a  mechanism  for  matching  these  functions  and  requirements. 


MEASURES 

Measures  (Figure  3)  are  the  graphic  displays  of  the  outputs  of  the  NWC 
activities  or  functions.  Measures  are  subdivisions  within  NWC  activities 
that  best  reflect  each  activity.  Each  activity  can  have  several  measures.  See 
Appendix  B  for  a  discussion  of  suggested  measures  appropriate  for  a 
research -and -development  environment. 
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MEASURES 


FIGURE  3.  Measures. 


Superimposed  on  any  measure  graph  is  the  desired  target  value 
(Figure  4).  Target  values  are  goals  that  are  mutually  and  interactively 
established  between  the  customers  and  NWC.  Selection  of  a  target  value 
provides  a  way  of  establishing  a  baseline,  demonstrating  the  gap  between 
what  is  and  what  should  be,  and  tracking  changes  and  improvements  over 
time.  Therefore,  measures  become  graphic  displays  of  the  degree  to  which 
conformance  to  customer  requirements  is  being  effected. 


FIGURE  4.  Typical  Measure  With  Target  Value. 
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OWNERS 


An  NWC  owner  (Figure  5)  is  the  individual  (or  small  group)  with  the 
primary  responsibility  for  an  NWC  activity  that  reflects  a  particular 
customer  requirement.  The  owners  are  responsible  and  accountable  for  the 
many  intermediate  products  and  processes  that  eventually  are  combined 
into  the  collective  products  of  NWC  as  a  whole.  The  daily  output  of  owner" 
produces  the  information  that  is  reflected  in  the  graphic  measures.  Each 
owner  contributes  in  a  different  way.  See  Appendix  C  for  a  discussion 
about  criteria  for  choosing  measures  and  how  owners  can  contribute. 


NWC  PRODUCTS  AND  PROCESS  OWNERS 


CHINA  LAKE-PEOPLE  COMMITTED  TO  EXCELLENCE 
THROUGH  TEAMWORK  AND  INNOVATION 


FIGURE  5.  C  vners. 

Individual  commitment  to  excellence  is  the  turning  point  that 
transforms  the  flow-down  part  of  mission  and  policy  deployment 
throughout  the  organization  into  the  flow-up  part  of  quality  improvement. 

SURVEYS 

Sample  surveys  provide  data  for  assessing  attitudes  and  making 
decisions  (Dunham).  Properly  designed  surveys  capture  information  that 
reflects  preferences  of  those  queried  (Fienberg).  Appendix  D  contains 
additional  information  concerning  what  constitutes  a  good  survey,  the  use 
of  surveys  by  Florida  Power  and  Light,  and  additional  sources  of 
information  about  particular  types  of  surveys. 


QUALITY  TABLES 

The  primary  Quality  Table  (Table  1)  shows  a  suggested  categorization 
of  customer  requirements.  Table  2  shows  a  suggested  breakdown  in  two 
categories  with  sample  activities  including  measures  that  represent  these 
sample  activities.  Supplementary  tables  can  be  used  for  the  customer  to 
weight  the  requirements  and  for  NWC  to  rank  each  customer  by  an 
importance  factor  according  to  how  much  business  or  influence  that 
customer  has. 
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TABLE  1.  4  Typical  Quality  Table. 


Category 

Customer 

Rcuuirements 

NWC 

Activities 

Measures 

Quality 

Accurar 

Time'  <'  > 

CompL  -ss 

Clarity 

Accountability 

Responsiveness 

Effectiveness 

Efficiency 

Human  factors  engineering 

Deliverables 

Functional  performance 
Confon.iance  to  specifications 
Technical  flexibility 

Support 

Durability 

Affordability 

Producioil'ty 

Reliability 

Maintainability 

Environmental  engineering 
Susceptibility 

Vul:  erability 

Cost 

Management 

Containment 

Innovation 

Efficiency 

Productivity 

Safety 'Security 

Public  safety 

Employee  safety 

Customer  safety 

Product  security 

Ceporate 

itesponsibility 

Quality  of  worklife 

Concern  for  physical  environment 
Concern  for  local  communiues 
Financial  integrity 

Ethical/legal  integrity 

Quality  tables  can  be  as  simple  or  as  complicated  as  necessary.  A 
sophisticated  table  and  software  could  be  developed  that  would  combine 
all  correlation,  weighting,  and  ranking  factors  into  one  function.  An 
interactive,  user-friendly  database  could  be  constructed  so  that  employees 
involved  with  a  particular  product  could  acce.  s  the  data,  contribute 
information,  and  get  feedback. 
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TABLE  2.  Sample  Breakdown  Within  Quality  Table. 


Category 

Customer 

Requirements 

NWC 

Activities 

Measures 

IDeliverables 

Reliability 

Design  analysis 

Existence  of  fail-safe  systems 

Mean  time  between  failures 

Mean  time  to  first  failure 

Failures  per  unit  time 

Environmental 
stress  screening 
and  acceptance 
testing 

Infant  mortality  rate 

Ratio  of  production  failures  to  customer 
failures 

Parts  control 

No.  of  redesigns  needed  for 
nonprocurable  parts 

No.  of  "off-Center"  problem-solving  trips 

Others 

Maintainability 

Requirements 

determination 

Mean  time  to  repair 

Ratio  of  maintenance  workhours  per 
operating  hour 

Unplanned  downtime 

Other  activities 

Corporate 

Responsibility 

i 

Eihical/legal  integrity 

i 

EEO  issues 

EEO  Complaints 

-  No.  of  informal  complaints 

-  No.  of  resolutions 

-  No.  of  formal  complaints 

Affirmative  action 

-  No.  of  minority/womcn  hired 

-  No.  of  handicapped  hired 

-  No.  of  minority/womcn  promotions 

-  No.  of  handicapped  promotions 

-  Special  emphasis  programs 

-  Initiatives  in  the  special  emphasis 

programs 

Other  rssucs 
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SELECTING  PRIORITIES 


Prioritizing  customer  requirements  is  done  through  information 
obtained  from  customer  surveys  and  from  expert  judgement  within  NWC. 
Customers  can  weight  requirements  according  to  their  individual 
preferences.  Combining  weights  yields  a  list  of  important  requirements  in 
descending  order.  NWC  can  rank  its  customers  by  an  importance  factor 
that  numerically  reflects  the  amount  of  product  use  by  a  customer  or  the 
influence  that  customer  has  over  the  Center's  business. 

This  competitive  evaluation  of  customer  requirements  yields  pooled 
numerical  results  that  indicate  the  best  possible  resource  allocation  for 
gaining  the  most  impact  toward  improving  products  and  services. 

The  entire  collection  of  categories,  activities,  and  measures  stays 
basically  the  same  after  it  is  defined.  Prioritizing  the  customer 
requirements  yields  the  few  activities  that  are  selected  for  immediate 
improvement  and  points  to  their  representative  measures  that  can  be 
closely  monitored. 


MEASURES  OF  EFFECTIVENESS 

The  measures  of  effectiveness  are  those  few  measures  that,  through 
the  Surveys  and  priority  selection  just  discussed,  have  been  selected  as 
best  representing  the  priority  customer  requirements.  These  external  and 
internal  quality  measures  represent  the  "vital  signs"  of  NWC.  Focusing 
common  efforts  on  improving  these  few  measures  yields  the  most  impact 
towards  improving  the  quality  of  NWC  products. 

These  measures  can  be  augmented  by  other  measures  as  a  more 
comprehensive  reflection  of  quality  efforts.  Appendix  E  contains  details. 


SUMMAT.Y 

Customer  requirements  are  weighted,  ranked,  and  prioritized 
according  to  the  customer  surveys  and  expert  judgement  within  NWC.  The 
customers'  preferences  are  taken  into  consideration,  and  conflicting 
impacts  within  NWC  are  also  addressed. 
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The  appropriate  categories  and  NWC  activities  that  best  represent  the 
priority  requirements  are  identified.  The  measures  within  these  activities 
that  best  reflect  the  customers'  priorities  are  selected  for  immediate 
improvement.  Short-term  (and  long-term)  targets  are  set  for  these 
measures.  These  few  measures  become  the  priority  list  of  quality 
measures  (or  measures  of  effectiveness),  because  improvements  in  these 
selected  few  should  provide  the  most  impact  towards  improving  customer 
satisfaction  (Figure  6). 

The  overall  matrix  of  information  tracked  (categories,  NWC  activities, 
and  measures)  stays  basically  the  same.  All  measures  continue  to  be 
monitored,  even  though  only  a  few  need  immediate  improvement.  As  soon 
as  priority  targets  are  reached,  these  measures  drop  out  of  the  priority  list, 
and  others  are  selected  as  priorities. 


FIGURE  6.  Generation  of  Measures  of  Effectiveness. 
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EVALUATION 


VALIDATION 

Florida  Power  and  Light  (FP&L)  has  for  several  years  successfully  used 
the  model  described  in  this  report.  FP&L  is  the  first  non-Japanese  company 
to  be  awarded  the  Deming  Prize  by  the  Japanese  Union  of  Scientists  and 
Engineers.  This  award  signifies  the  degree  to  which  the  Japanese  were 
impressed  with  FP&L's  ability  to  implement  and  maintain  TQM.  FP&L's 
measurement  model  has  matured  over  the  past  few  years  into  a  very 
sophisticated  "table  of  tables"  for  capturing  the  customer  voice  and 
translating  it  into  business  activities  (Qualtec). 

Historical  data  are  available  for  most  of  the  measures,  so  such  data  are 
amenable  to  formal  statistical  analysis  any  time  the  need  should  arise. 
Baselines  for  new  measures  can  be  established. 

Although  the  assignment  of  priorities  seems  subjective,  it  is  believed 
by  experts  that  even  subjective  results  can  become  objective  standards 
that  are  measurable  and  statistically  assessable  (Garvin,  Reid,  Salvendy). 

If  the  customer  surveys  are  performed  properly,  the  data  gathered 
are  also  appropriate  for  formal  statistical  analysis  (Fienberg).  One  can 
study  trends,  significant  new  events,  and  comparisons  of  new  standards 
with  previous  standards.  One  can  also  gain  additional  insight  by  exploring 
a  multitude  of  relationships  in  the  information. 

LIMITATIONS 

The  measurement  model  requires  the  development  of  a 
comprehensive  and  consistent  method  for  surveying  customer  needs.  This 
is  a  task  that  is  routinely  performed  by  many  organizations,  and  there  is  a 
wealth  of  expert  information  available  from  many  sources  to  assist  in  the 
development  of  valid  and  reliable  survey  models  (Salvendy,  Jessen).  See 
Appendix  D  for  more  information. 

Other  modules  of  TQM  need  to  be  in  place  for  most  efficient  use  of  the 
measurement  model's  capabilities.  One  necessary  ingredient  is  an  efficient 
employee-suggestion  system.  Another  is  training  throughout  NWC  in 
several  areas,  such  as  systems  engineering,  mathematical  modeling,  and 
statistical  training,  to  supplement  TQM. 


14 


Commitment  "to  excellence  through  teamwork  and  innovation"  is  the 
cornerstone  that  makes  TQM  really  work  within  this  measurement  model. 
However,  scientific  analysis  and  applications  tools  such  as  statistical 
process  control,  Taguchi  methods,  and  other  graphic  problem-solving 
techniques  need  to  be  in  place  to  capture  and  anchor  individual 
commitment.  Availability  of  these  applicaiions  encourages  proactive 
participation  at  all  levels. 

One  frequent  criticism  of  applying  TQM  in  a  research-and- 
development  environment  concerns  the  difficulty  of  quantifying 
productivity  and  creativity  of  scientists  and  engineers.  To  a  degree,  this 
criticism  is  valid.  However,  some  approaches  have  been  suggested  in 
recent  literature.  Additionally,  there  are  many  ways  to  describe  the  factors 
that  have  contributed  to  the  past  and  present  success  of  NWC  (NWC,  1968; 
NWC,  August  1990).  This  wealth  of  information  is  readily  available  for 
examining  our  processes  in  more  detail  and  identifying  measures  that  can 
indeed  represent  or  reflect  creativity  and  productivity  in  an  R&D 
environment.  Several  approaches  are  described  in  Appendix  F. 


CONCLUSIONS 


The  measurement  model  described  in  this  report  and  shown  in  Figure 
7  on  page  39  represents  a  mechanism  for  more  efficient  deployment  of  the 
NWC  mission  statement  throughout  the  Center.  The  model  promotes  the 
actualization  of  TQM  by  redefining  activities  within  the  organization  in 
such  a  way  that  management  captures  the  voice  of  the  customer.  Priorities 
flow  down  into  sublevel  functions,  NWC  resources  are  focused  on  the 
selected  priorities,  and  employees  know  what  is  expected.  Thus  everyone 
in  the  organization  has  the  opportunity  to  contribute  immediately  and 
positively  to  improvements  that  flow  up  into  customer  satisfaction  wuh 
NWC  products  and  services. 

The  benefits  of  accepting  and  applying  this  measurement  model  are 
potentially  enormous.  It  provides  a  mechanism  through  which  "the  voice  of 
the  customer  is  captured  and  translated  into  the  way  business  is  done" 
(Qualtec)  or,  stated  another  way,  through  which  customer  requirements 
are  equated  to  product  characteristics  (Reid).  Through  this  model,  TQM  can 
be  implemented  with  specific  tools  and  perpetually  maintained.  The 
measurement  model  provides  an  architecture  within  which  NWC  can 
continually  meet  its  customer  needs  and  thereby  guarantee  continuing 
recognition  as  a  quality  organization. 
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RECOMMENDATIONS 


It  could  be  beneficial  for  NWC  and  Qualtec,  Inc.,  to  develop  a  working 
relationship  for  the  exchange  of  information,  ideas,  and  expertise.  Florida 
Power  and  Light  learned  TQM  methods  directly  from  the  Japanese  and  has 
successfully  practiced  TQM  for  several  years.  The  consultants  at  Qualtec 
are  some  of  the  key  personnel  who  nurtured  FP&L  employees  through  the 
transformation  into  an  internationally  recognized  quality  organization. 
NWC  has  achieved  success  in  areas  that  are  of  interest  to  Qualtec,  such  as 
the  extensive  use  and  networking  of  personal  computers. 

The  author  recommends  that  every  effort  be  made  to  implement  the 
quality  measurement  model  described  in  this  report.  An  ideal  approach 
would  be  to  have  a  highly  analytical  individual,  with  the  ability  to  relate  to 
all  levels  of  Center  organization,  oversee  and  implement  an  integrated 
effort  that  includes  the  iterative  processes  of  customer  surveying  and 
prioritizing  objectives  within  the  Quality  Table.  A  core  group  could  be 
formed  with  representatives  from  each  category  with  collective 
backgrounds  in  necessary  disciplines  such  as  mathematics,  statistics, 
business,  and  engineering.  This  approach  would  help  to  ensure  that 
surveying  and  prioritizing  would  overlap  all  functional  areas  within  NWC. 

Implementation  of  this  measurement  model  through  a  concerted  team 
effort  by  NWC  workers  and  management  alike  under  the  leadership  )f  a 
committed  individual  will  help  NWC  evolve  to  a  more  advanced  level  of 
TQM.  With  implementation  of  this  model,  NWC  will  continue  to  be 
recognized  as  a  master  of  innovation  and  successful  implementation  and  as 
a  leader  in  sharing  successful  approaches  with  DOD  counterparts. 
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Appendix  A 

CUSTOMER  REQUIREMENTS 


Customer  requirements  are  the  needs  and  wants  of  the  customer 
expressed  in  tf*rms  of  mutually  agreed  upon  specifications.  Communication 
with  customers  should  produce  a  list  of  valid  requiiernents  that  are 
current,  realistic,  and  measurable,  that  meet  reasonable  expectations  of  the 
customer;  and  that  fall  within  corporate  constraints.  Requirements  will 
generally  address  areas  of  concern  like  performance,  conformance, 
durability,  affordability,  cost,  and  others.  A  list  of  typical  customer 
requirements  with  definitions  or  qualifiers  is  set  out  below. 

Accountability:  Acceptance  of  responsibility 

Accuracy:  Processing  correctly 

Affordability:  Within  certain  cost  constraints 

Availability:  Probability  that  a  system  is  in  operable  condition  at  the 
start  of  a  mission;  a  function  of  Reliability  and  Maintainability; 
(a  sample  measure:  the  incidence  of  unplanned  downtime  (NWC 
1980)) 

Clarity:  Understandability,  clearness  of  statements 

Completeness:  Thoroughness;  no  missing  parts  or  information; 
percentage  of  critical  issues  addressed 

Concern  For  Local  Communities:  Corporate  sensitivity  to  the 
impact  of  NWC  on  local  comm.unity  resources;  interaction  with 
local  governments;  contribution  to  local  charities  and  volunteer 
activities 

Concern  for  Physical  Environment:  Corporate  concern  for  land, 
endangered  species,  water,  waste  management,  etc. 

Confoimance  to  Specifications:  Degree  to  which  design 

characteristics  and  operating  characteristics  meet  established 
standards  (Garvin);  (sample  measures:  acceptable  defect  rates 
after  product  is  in  use,  tolerance  "stack-up"  costs,  Taguchi  loss 
function) 

Cost  Containment:  Control  of  costs 

Cost  Efficiency:  Ratio  of  output  produced  per  cost  (Byrns) 

Cest  Management:  Attention  to  the  budget 
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Customer  Safety;  Provision  of  reasonably  safe  working  and  handling 
environment  for  the  end  user  of  NWC  products 
Durability:  Expected  product  life  (Garvin) 

Effectiveness;  Ratio  of  actual  milestones  to  milestones  set  for  a  given 
time  period  (Somers) 

Efficiency;  Ratio  of  planned  resources  to  consumed  resources 
(Somers) 

Employee  Safety:  Provision  of  relatively  safe  work  environment 
Environmental  Engineering:  Designing  items  to  function  in  the 
operational  environment  (NWC,  Januaryl990) 

Ethical  Integrity:  Understanding  and  working  according  to  a  strong 
set  of  principles  and  communicating  these  principles  to  all  aspects 
of  the  organization  and  the  community  at  large 
Financial  Integrity:  Soundness  of  proper  financial  transactions; 
maintaining  accounts  in  a  way  that  meets  the  letter  and  spirit  of 
accepted  fiscal  standards 

Functional  Performance;  Matching  products  to  user  needs; 

addresses  the  basic  functions  of  the  product 
Human  Factors  Engineering:  Design  of  items  to  produce  effective 
human-machine  integration  (Dhillon,  1986) 

Innovation;  Creative  changes  and  adaptations  (Somers);  creative  and 
effective  engineering  approaches;  (sample  measures:  savings, 
generating  unexpected  revenue) 

Maintainability:  Ability  to  retain  in  (or  restore  to)  a  specified 
condition  with  prescribed  maintenance  (NWC,  January  1990); 
addresses  ease  and  speed  of  preventative  maintenance, 
serviceability,  repairability,  availability  of  spare  parts,  etc.; 
(sample  measures  could  be  mean  time  to  repair,  maintenance 
work  hours  per  operating  hour,  unscheduled  downtime) 
Producibility :  Enables  an  item  to  be  manufactured  and  inspected  at 
the  desired  rate;  reducing  production  costs  while  maintaining 
quality  (NWC,  January  1990) 

Product  Security:  Ability  to  protect  the  product  from  unauthorized 
use  or  access;  customer's  ability  to  control  the  product  after  it  is  in 
use;  protection  of  documentation,  reports,  and  processing 
procedures  (especially  software)  from  direct  or  indirect  damage  or 
errors 

Productivity:  any  ratio  of  outputs  to  inputs  (Knudson) 

Public  Safety:  Minimizing  danger  to  the  general  public 
Quality  of  Work  Life:  Health,  wellness,  training,  career  mobility,  job 
satisfaction,  open  communication,  etc. 

Reliability:  Probability  that  an  item  will  perform  its  intended 
function  for  a  specified  interval  under  stated  conditions  (NWC, 


January  1990);  (sample  measures:  "infant  mortality,"  existence  of 
fail-safe  systems,  mean  time  between  failures,  mean  time  to  first 
failure,  failure  rate  per  unit  time,  ratio  of  production  failures  to 
customer  failures  (Garvin)) 

Responsiveness:  Ready  response  of  NWC  personnel  to  an  appeal  for 
aid  by  a  customer 

Support:  Product-improvement  support,  phase-in  support,  operations 
support,  and  phase-out  support  (NWC,  1980) 

Survivability;  Ability  to  carry  out  a  mission  in  a  hostile 
environment;  function  of  both  Susceptibility  and  Vulnerability 
(NWC,  1980) 

Susceptibility:  Probability  of  damage  by  a  threat  (NWC,  1980) 

Technical  Flexibility:  Features  that  supplement  basic  features; 
ability  to  customize  to  more  specific  user  needs  (Garvin);  flexible 
technology;  portability  (especially  in  software);  for  example,  use  of 
off-the-shelf  components 

Timeliness:  Promptness  of  delivery 

Vulnerability:  Extent  of  system  degradation  after  subjection  to 
threats  (NWC,  1980) 
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Appendix  B 

TYPICAL  MEASURES  OF  RESEARCH  PRODUCTIVITY 


The  National  Science  Foundation  routinely  uses  publication  and  patent 
information  as  a  measure  of  research  productivity  and  regularly  reports 
how  various  segments  of  engineering,  academic  research,  industrial 
research,  etc.  stand  with  respect  to  national  and  international  standards 
(National  Science  Board).  It  is  important  to  know  if  any  of  these  typical 
measures  are  used  by  Navy  or  DOD  agencies  for  awarding  funds  or 
projects,  or  for  comparing  NWC  to  similar  organizations. 

PATENT  APPLICATIONS 

Tracking  the  number  of  patents  awarded  can  lead  to  distorted 
information  about  the  time  span  actually  represented  by  the  efforts 
leading  up  to  the  application.  Tracking  the  number  of  patent  applications 
sometimes  can  be  a  better  reflection  of  the  time  in  which  the  research 
occurred.  Also,  shared  patents  can  be  a  measure  of  cooperative  research 
efforts.  Other  types  of  cooperative  agreements  can  also  be  tracked. 

PUBLICATION  EFFORTS 

New  knowledge,  the  primary  product  of  research,  can  be  measured  by 
various  publications  counts  (National  Science  Board)  External  refereed 
(peer-reviewed)  articles,  other  publications,  and  presentations  (to  some 
minimum  level  of  audience)  are  all  indicative  of  research  output.  For 
instance,  tracking  numbers  of  citations  provides  an  indication  of  how  much 
impact  an  individual  report  has  on  other  research. 

EDUCATION 

The  National  Science  Foundation  tracks  detailed  information  about  the 
status  of  education  in  all  sectors  of  American  industry,  research,  and 
academia.  This  information  is  used  to  predict  future  R&D  productivity.  The 
Foundation  also  makes  detailed  observations  about  national  and 
international  trends,  changes,  and  comparisons  of  groups  to  each  other. 
Several  potential  NWC  measures  in  this  area  are  listed  below. 
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Level  of  Education 


Average  education  level  of  scientists  and  engineers  can  be  an 
indication  of  the  sophistication  of  problem  solving  capability. 


Grade  Point  Average 

Average  grade  point  average  (GPA)  of  incoming  Junior  Professionals 
can  be  a  measure  of  future  capability  and  success.  (Alternative  measures 
could  be  developed  to  identify  special  capabilities  among  those  with 
mediocre  GPAs.) 

Continuing  Education 

Number  of  current  education  endeavors  (especially  in  areas  identified 
as  critical)  is  an  indication  of  interest  level  and  desire  for  better 
technology.  Among  the  areas  that  need  continuing  training  are  systems 
engineering  approaches,  mathematical  modeling  as  an  enhancement  to 
one  s  own  technical  field,  and  statistical  training  to  supplement  TQM. 

INTEGRATED  APPROACHES 

Integrated-discipline  efforts  enhance  elegant  problem  solving, 
especially  in  software  development.  For  example,  some  problems  could  be 
elegantly  attacked  by  a  small  team  with  an  engineer,  a  computer  scientist, 
and  an  applied  mathematician. 
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Appendix  C 

CRITERIA  FOR  CHOOSING  MEASURES 


TYPES  OF  MEASURES 
Direct  Measures 

Direct  measures  are  vertical  measures.  They  directly  add  up 
numerically.  Everyone  measures  the  same  units.  For  example,  if  the  goal  is 
to  reduce  deaths  and  accidents,  then  everyone  contributes  the  same 
information:  counts  of  deaths  or  accidents.  Direct  measures  tend  to  be  more 
common  at  high  levels  in  an  organization. 

A  useful  example  is  taken  from  FP&L.  Given  the  goal  of  restoring 
power  quickly,  employees  are  sent  to  check  lines,  substations,  and  other 
equipment.  Different  groups  are  contributing  the  same  type  of  information, 
which  is  directly  additive. 

Indirect  Measures 

Indirect  measures  are  horizontal  in  nature.  These  measure  the  effects 
of  different  activities  and  eventually  become  a  system  of  measures  that 
contributes  effects  that  are  cumulative,  but  not  directly  numerically 
additive.  Indirect  measures  tend  to  be  more  common  at  group  or 
individual  activity  level. 

If  the  goal  is  to  produce  accurate  bills,  different  groups  contribute 
different  information.  For  example,  at  FP&L,  meter  readers  will  gather 
information  about  access  to  meters,  data  gathering  errors,  etc.,  whereas 
accounting  clerks  will  address  issues  concerning  the  database,  training 
adequacy,  and  complications  in  the  pricing  schedule.  Different  groups 
monitor  different  things,  but  the  cumulative  effects  are  considerable  when 
examined  all  together. 
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Auditability 


Auditability  means  that  anyone  can  measure  the  same  thing  in  the 
same  units.  There  is  agreement  as  to  the  quantification  of  some 
characteristic.  Quantification  is  tailored  to  the  specific  requirements. 
Information  from  different  samples  at  different  times  can  be  meaningfully 
compared. 


TOP-DOWN  APPROACH 

Priorities  for  improvement  and  the  choice  of  measures  should  be 
determined  at  the  top  level  of  an  organization,  and  these  top-down 
strategic  issues  should  be  communicated  throughout  the  organization. 
Therefore,  sublevel  measures  are  partially  predetermined.  Each  sublevel 
either  tracks  a  measure  in  the  same  units  already  or  can  choose  what 
contributes  indirect  effects  to  a  particular  measure. 

Corporate-ievel  measures  should  represent  a  balance  of  in-house 
expertise;  types  of  customers;  and  issues  that  surpass  the  customers' 
expertise,  like  business  forecasting  and  regulatory  requirements.  This 
approach  takes  into  account  all  the  important  factors  for  making  decisions 
and  setting  priorities. 

Consistency  and  Flexibility 

Measures  need  to  be  consistently  applied  yet  flexible.  Measures  can  be 
tracked  at  the  corporate  level  and  at  sublevels  either  in  direct  measures  or 
indirect  effects  on  a  particular  measure.  This  flexibility  allows  everyone  to 
see  a  way  to  contribute  to  improvement.  For  example,  if  safety  is  the  issue, 
some  groups  can  count  actual  accidents  (direct)  and  others  can  show 
efforts  at  preventing  accidents  (indirect). 

Attitudes  (pride,  for  example)  should  be  anchored  into  an  analytic 
measure.  Employees  should  focus  on  something  that  affects  customer 
satisfaction,  like  eliminating  bottlenecks  to  progress.  When  dealing  with  a 
"local  way"  that  seems  better  than  the  way  others  do  business,  it  is 
important  to  show  its  efficiency  and  possibly  get  official  acknowledgment. 
That  is,  a  subtle  way  to  measure  the  uniqueness  of  an  organization  is  to 
track  its  ability  to  transport  better  methods  to  other  groups. 
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General  Sequence  of  Events 

Initially,  in-house  expertise  is  used  to  determine  what  is  important  to 
the  customer.  Emphasis  should  be  on  defining  processes  within  the 
organization  and  understanding  functions  within  the  matrix  of  interaction 
of  events  for  the  measures.  That  is,  an  organization  should  not  try  to 
benchmark  itself  against  others  too  soon. 

Surveying  customer  requirements  helps  to  continuously  refine  the 
method  of  choosing  measures.  Management  needs  to  communicate  the 
evolving  priorities,  and  all  levels  need  to  support  the  choice  of  measures  to 
improve.  The  development  of  a  good  chain  of  related  events  demonstrates 
knowledge  about  what  factors  contribute  to  any  measure.  Occasionally, 
new  technology  contributes  new  measures  to  the  system. 

The  priority  measures  evolve  with  changing  priorities  of  the 
customers.  The  system  of  categories,  activities,  and  measures  stays 
basically  the  same  after  the  whole  system  is  defined,  but  the  priority 
measures  slowly  change  as  results  from  customer  surveys  are  integrated 
into  the  TQM  measurement  model.  Some  measures  will  drop  out  of  the 
priority  list  (even  though  they  continue  to  be  tracked)  and  other  measures 
will  end  up  in  the  priority  list.  Many  measures  are  tracked  and  each  has  its 
own  set  of  contributing  factors,  but  surveys  of  customer  requirements  and 
the  NWC  analysis  of  the  results  determine  which  measures  are  chosen  as 
members  of  the  priority  list  and  how  long  they  stay  priorities. 

Measures  that  are  customarily  tracked  are  sorted  into  short-term  and 
long-term  goals  and  can  evolve  into  and  out  of  the  corporate  list  of  priority 
measures  of  effectiveness  as  customer  focus  changes  and  complaints 
change.  That  is,  when  a  priority  customer  requirement  dictates  the 
improvement  of  its  best  representative  measure,  then  this  measure  is  now 
in  the  priority  list  of  measures  of  effectiveness.  Short-term  and  long-term 
goals  are  set.  When  these  goals  are  reached,  the  selected  measure  drops 
out  of  the  priority  list  and  is  replaced  by  another  that  has  drawn  more 
recent  customer  concern.  This  corporate  list  of  priority  measures  can  be 
called  the  periodic  measures  of  effectiveness  for  NWC. 

This  method  provides  a  fluid,  dynamic,  and  flexible  system  for 
choosing  priority  measures.  Customer  needs  are  translated  into 
requirements.  The  organization  defines  what  functions  best  respond  to  the 
customer  requirements.  Some  requirements  focus  directly  on  what  the 
customer  wants,  but  others  need  to  be  modified.  That  is,  customer 
complaints  need  to  be  translated  into  what  can  be  measured  and  what  will 
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contribute  most  to  customer  satisfaction.  For  example,  FP&L  customer 
surveys  showed  that  customers  wanted  continuity  of  electric  service.  It 
took  FP&L  some  time  to  decide  that  focusing  efforts  on  reducing  unplanned 
shutdowns  contributed  the  most  to  customer  satisfaction.  Therefore, 
reducing  "customer  minutes  interrupted"  became  the  priority  issue  with 
FP&L. 


Not  all  measures  need  improvement,  but  most  need  to  be  maintained 
at  a  certain  level  or  quality  of  output.  Avoiding  complacency  and 
maintaining  processes  that  have  already  been  improved  is  equally  as 
important  as  the  original  improvement  efforts.  Maintaining  established 
quality  levels  provides  an  additional  avenue  for  employees  to  contribute  to 
an  overall  quality-improvement  effort. 
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Appendix  D 
SURVEYS 


COMPONENTS  OF  A  GOOD  SURVEY  MODEL 

The  first  consideration  when  putting  together  a  survey  is  a  good 
design.  Well-thought-out  questions  are  a  must,  including  understanding  of 
what  the  question  really  asks  and  of  the  respondent  audience  that  is  being 
queried.  Well-structured  questions  capture  the  right  information.  For 
example,  sets  of  revolving  questions  can  help  establish  priorities.  Open- 
ended  questions  can  allow  the  respondent  to  state  stronger  opinions.  Also, 
open-ended  questions  can  solicit  unexpected  responses  that  could  not  have 
been  anticipated  (Dillman). 

The  second  consideration  is  good  analysis  of  a  survey.  The  procedures 
for  processing  survey  data  should  be  designed  and  established  well  before 
the  data  are  collected  (Dunham).  Careful  codifying  of  the  results  is 
important.  It  is  important  to  understand  validity  and  reliability  of  survey 
results.  A  valid  question  measures  what  it  is  supposed  to  measure, 
whereas  the  reliability  of  a  measure  is  its  consistency  in  giving  the  same 
reading  (Patchen).  The  treatment  of  response  errors  should  be  understood 
(lessen). 

The  third  consideration  is  good  documentation.  Common  sense  dictates 
that  good  documentation  is  essential  in  any  analytic  investigation. 

A  well-designed  survey  can  yield  results  that  reflect  overall  attitudes. 
A  good  survey  can  demonstrate  what  employees  think  about  quality- 
improvement  efforts,  what  employees  think  about  supervisors’  attitudes 
and  vice  versa,  and  what  employees  think  about  quality  of  work  life.  Most 
importantly,  for  purposes  of  this  report,  customer  attitudes  about  product 
requirements  can  be  assessed. 

The  results  of  surveys  can  be  used  to  improve  products,  improve 
customer  relations,  and  provide  a  basis  for  direct  communication  between 
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the  customer  and  the  employee  most  responsible  for  the  product 
characteristics.  Also,  studying  the  results  can  help  improve  the  survey 
questions  as  the  survey  model  matures.  Survey  results  not  only  give 
immediate  information  about  what  is  important  to  the  customer,  but  also 
monitor  changes  in  customer-satisfaction  levels  over  time. 


DEVELOPMENT,  ADMINISTRATION,  AND 
ASSESSMENT  OF  SURVEYS  AT  FP&L 

Florida  Power  and  Light  prefers  outside  consultants  for  survey  activity 
(Qualtec).  FP&L  helps  formulate  the  questions,  but  Walker  Research  has 
the  primary  responsibility  for  the  surveying  activities  (Walker).  FP&L 
believes  that  the  use  of  outside  consultants  provides  a  sound  survey 
instrument  and  demonstrates  sincerity  on  the  part  of  FP&L  towards 
customers  and  employees  for  their  perspectives  and  opinions. 

The  use  of  outside  consultants  also  helps  eliminate  the  biases  of 
internal  analysts.  Another  benefit  of  employing  outside  consultants  is  that 
doing  so  provides  anonymity,  especially  for  internal  employee  surveys, 
and  therefore  yields  much  useful  information.  A  parallel  to  this  policy  is 
the  way  in  which  NWC  uses  an  outside  consultant  for  career  testing  and 
counseling  of  Junior  Professionals. 

FP&L  performs  three  basic  types  of  surveys.  For  each,  samples  are 
selected;  exhaustive  surveying  is  not  done.  A  customer-requirements 
survey  is  performed  yearly  for  determining  problem  areas,  setting  goals, 
and  choosing  priority  measures.  Customer-satisfaction  surveys  are 
administered  quarterly  to  give  periodic  assessment  as  to  whether  or  not 
there  has  been  improvement.  Employee  surveys  are  given  once  or  twice  a 
year  and  are  three  to  four  pages  long  with  many  simple  questions.  One  of 
the  areas  of  concern  that  the  employee  surveys  are  designed  to  assess  is 
the  employees'  attitudes  about  organizational  commitment  to  quality 
efforts. 


FURTHER  SOURCES  OF  SURVEY  INFORMATION 

1.  Webber  Group  (708)-653-0583  and  Standard  &  Associates,  Inc.  (312)- 
337-1729;  Quality  Climate  Survey. 

The  two  groups  have  combined  efforts  to  help  companies  determine 
the  level  of  employee  receptiveness  to  quality  improvement  efforts.  The 
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Climate  Survey  addresses  three  aspects  of  employee  attitude;  company 
commitment  to  quality,  degree  of  employee  participation,  and  general 
employee  attitude.  More  than  50%  of  the  survey  is  drawn  from  the  SRA 
(Science  Research  Associates)  Attitude  Survey  System,  allowing  companies 
to  compare  their  employee  responses  to  national  norms.  The  Climate 
Survey  can  be  used  as  part  of  quality  audits,  to  identify  areas  of  resistance 
to  change,  and  as  a  quantitative  means  to  increase  the  importance  of 
quality  in  performance  appraisal  systems  (^Quality  Progress,  March  1990.  p. 
15). 

2.  General  Research  Corp,  Technical  Support  Group,  7655  Old  Springhouse 
Road,  Mclean,  Va.  22102;  "Development  of  Productivity  and  Quality  Self- 
Evaluation  Audits,"  Report  #1582-03-89-CR,  September  89;  authors 
Somers,  Rouiller,  Hsiao. 

This  report  describes  the  development  of  the  audit,  the  tests  of  the 
audit  at  six  DOD  organizations,  the  results  of  the  tests,  and  an  assessment  of 
the  audit's  validity  and  internal  reliability. 

3.  William  H.  Hendrix  (Clemson  University).  "Development  of  a 
Contingency  Model  Organizational  Assessment  Survey  for  Management 
Consultants,"  for  use  by  Air  Force  management  consultants  at  the  Air  Force 
Leadership  and  Management  Development  Center  (LMDC);  Journal  of 
Experimental  Education,  Winter  1984,  pp.  95-105. 

Organizational  effectiveness  can  be  considered  the  degree  to  which 
managerial  goals  are  attained.  The  organizational  assessment  package  was 
developed  within  a  contingency  model  of  organizational  effectiveness  and 
is  a  diagnostic  instrument.  A  contingency  approach  to  organizational 
effectiveness  considers  effectiveness  to  be  a  function  of  the  manager,  the 
situational  environment,  and  the  criteria  of  success.  Criteria  of 
organizational  effectiveness  include  hard  or  objective  measures 
(productivity,  quality,  efficiency,  profit,  growth)  and  attitudinal  or 
subjective  measures  (morale,  motivation,  satisfaction).  The  report 
describes  the  package,  survey  questions,  factor  analysis,  results,  and 
limitations  of  the  model. 
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Appendix  E 

ADDITIONAL  MEASURES  OF  EFFECTIVENESS 


In  addition  to  the  priority  measures,  the  measures  of  f*ffectiveness 
might  routinely  include  two  other  measurements.  Firstly,  th^  overall 
percent  of  employee  participation  in  the  organizational  suggestion  system 
should  be  included.  It  represents  the  "heartbeat"  of  NWC.  Secondly,  the 

results  of  separate  surveys  that  represent  perceived  customer  satisfacti'~ 
could  be  reported.  These  results  represent  the  "pulse"  of  the  custom 
base.  Additionally,  the  measures  of  effectiveness  of  NWC  could  include  the 
results  of  a  general  survey  of  employees  that  demonstrate  the  collective 
perception  of  the  level  of  TQM  implementation  at  NWC.  These  results 
represent  the  "pulse"  of  the  employee. 

Florida  Power  and  Light  believes  that  the  degree  of  participation  in 
suggestion  systems  represents  the  collective  belief  of  individuals  in  the 

workability  of  the  overall  management  system,  a  theory  similar  to  some 
regarding  individual  investor  participation  in  the  stock  market  (Qualtec). 
At  the  corporate  level,  overall  percent  of  participation  is  measured. 
Management  reports  reflect  how  teams  and  groups  are  participating 
compared  to  each  other.  Measuring  the  number  of  ideas  per  generic  group 
is  better  than  measuring  per  individual.  (Although  FP&L  does  not  include 
this  information  in  individual  performance  evaluations,  the  Japanese  track 
individual  participation.)  Broad  participation  at  all  levels  and  across  all 

disciplines  represents  good  communication  within  an  organization.  At  each 
sublevel,  percent  of  participation  is  measured,  but  each  level  down  needs 
to  track  more  information  about  suggestions.  For  example,  at  local  level 
and  suggester  level,  turnaround  time  in  responding  to  suggestions  is 
important  for  maintaining  employee  faith  in  the  effectiveness  of  the 

system. 

Measuring  participation  in  social  events  can  be  a  reflection  of 
communication  at  all  levels  and  stratifications  within  an  organization.  If 
employees  believe  that  supervisors  participate  adequately  in 
organizational  social  and  wellness  events,  then  this  belief  can  be  a  subtle 
measure  of  leadership. 
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Appendix  F 


CREATIVITY  AND  PRODUCTIVITY  IN 
RESEARCH  AND  DEVELOPMENT 


Any  of  the  following  processes  could  be  defined  in  such  a  way  as  to 
identify  factors  that  contribute  to  improvement  efforts.  The  development 
of  scoring  models  (trade-off  studies  with  weighting  and  ranking  of  factors) 
as  decision  aids  is  also  possible  for  any  of  these  activities. 


RESEARCH  AND  DEVELOPMEN  f  MANAGEMENT 

One  approach  to  measuring  R&D  productivity  is  to  evaluate  how 
effectively  R&D  resources  are  managed.  The  following  methods  are  from 
Szakonyi. 


Ranking  of  Research  Projects 

A  model  needs  to  be  in  place  to  identify  which  research  projects  show 
the  most  promise.  The  best  trade-off  of  technical  versus  administrative 
requirements  for  prospective  projects  needs  to  occur  as  objectively  as 
possible.  It  is  also  important  to  have  an  effective  method  for  terminating 
projects  that  are  identified  as  probably  not  successful.  Terminating 
projects  should  occur  in  a  manner  that  captures  the  knowledge  generated 
so  that  ethers  do  not  repeat  the  same  efforts  and  so  that  individuals  are 
not  penalized  for  documenting  a  failure  (NWC,  1968). 


Evaluation  of  Project  Goals 

A  model  needs  *0  be  in  place  for  tracking  whether  projects  meet 
technical,  schedule,  and  cost  goals.  A  method  could  be  developed  for 
weighting  these  efforts  to  assess  output  of  research  projects  and  identify 
problems.  Such  an  evaluation  encourages  communication  and 
accountability. 
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Case  History  of  Projects 


It  is  important  to  document  the  development  and  successful 
application  of  particular  technologies.  This  historical  information  can  be 
used  to  predict  the  future  applicability  of  existing  technology.  Decisions  can 
be  made  about  whether  or  not  to  exploit  existing  technologies  or  develop 
new  ones. 


Evaluation  of  Research  Issues 

The  state  of  the  art  of  technology  within  an  organization  is  important. 
It  is  also  important  to  be  aware  of  technological  developmeiits  outside  of 
the  organization.  Future  needs  and  opportunities  within  the  organization 
need  to  studied.  It  is  important  '.o  have  a  "fast-turnaround"  method  for 
assessing  space,  equipment,  and  people  needs  for  a  new  project  under 
consideration.  Examining  the  balance  of  distribution  of  these  resources  is 
also  important.  Addressing  these  issues  would  link  the  best  potential  to  the 
most  desirable  goals.  The  National  Bureau  of  Standards  has  a  method  for 
addressing  some  of  these  issues  (Hall). 

REPLICATION  OF  UNIQUE  R&D  EFFORTS 

When  an  effort  results  in  an  outstanding  development  project,  there 
are  possibilities  for  defining  activities  in  which  measures  could  be 
developed. 


Repeatability  of  the  Effort 

If  the  genius  behind  an  effort  leaves,  the  project  should  not  lose  its 
flow  of  effectiveness.  The  expert's  brain  power  and  creativity  should  be 
developed  into  a  repeatable  effort.  The  same  group,  and  different  groups, 
should  be  able  to  replicate  the  effort. 


Transfer  Genius  Into  a  Standard 

A  successful  research-and-development  effou  should  be  transferred 
into  a  detailed  standard,  so  that  others  can  learn  from  a  creative  approach. 
For  example,  if  an  engineer  understands  how  not  to  approach  something  as 
well  as  how  to  do  it,  there  should  be  a  process  in  place  so  F'at  others  can 
take  advantage  of  that  knowledge  without  having  to  learn  by  repeating  the 
same  failures. 


CONTINUOUS  IMPROVEMENT  OF  R&D  EFFORTS 


Continuous  Improvement  Versus  Continuous  Replacement 

Some  NWC  activities  could  be  further  defined  to  ensure  that  products 
are  designed  in  such  a  way  that  continuous  improvement  is  reasonably 
possible  in  the  future.  Several  areas  that  could  be  considered  are 

•  Implementing  technical  advances  after  product  is  in  the  Fleet. 

•  Upgrading  less-than-perfect  parts  after  a  product  is  in  use.  (Even 
though  the  system  as  a  whole  is  good,  some  parts  could  have 
suffered  from  lack  of  attention  due  to  constraints.) 

•  Incorporating  new  technology  into  an  older  product. 

•  Reconfiguring  good  parts  or  subsystems  into  a  new  system  with  new 
capabilities. 

•  Directly  communicating  between  the  Fleet  user  and  NWC  so  that 
needs  are  known  and  improvements  are  appreciated,  and  this 
appreciation  is  communicated  directly  to  NWC  employees. 


Design  Criteria  Versus  Operating  Criteria 

One  important  issue  is  to  determine  to  what  extent  operating 
conditions  supersede  design  and  test  conditions.  For  example,  a  weapon 
system  may  be  designed  to  operate  well  once,  but  it  may  have  to  be 
primed  to  operate  several  times  in  drill  situations. 


Requirements  Versus  Restrictions 

It  is  important  to  achieve  the  best  possible  trade-off  between 
performance  requirements  and  restrictions.  For  example,  the  application  of 
Taguchi's  robust  design  helps  choose  a  reasonable  set  of  test  criteria  and 
optimize  design  parameters  to  ensure  that  the  product  will  meet  the 
requirements  of  the  customer  and  stay  within  constraints  (Chase).  A  robust 
design  (a  product  with  characteristics  that  are  tolerant  to  uncontrollable 
changes)  can  be  achieved. 


Improvements  Versus  Innovation 

It  is  important  to  determine  the  best  combination  of  significant 
improvement  without  technical  innovation  (efficiency  of  operation,  better 
maintenance,  better  vendor  selection,  better  documentation,  etc.)  versus 
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incorporation  of  new  technology.  This  determination  should  be  done 
through  a  systematic  logical  approach. 


DOCUMENTATION 

Documentation  efforts  reflect  efficiency:  information  ends  up  in 
retrievable  databases,  which  prevents  duplication  of  effort.  Documentation 
efforts  reflect  accountability:  funding  requests  and  expenditures  are 
accurately  described  and  explained.  Documentation  efforts  reflect 
reliability:  products  may  be  reliable,  but  if  problems  develop,  good 
documentation  can  be  the  key  to  quick  resolution.  Documentation  efforts 
reflect  sophistication:  in  addition  to  technical  expertise,  an  organization 
needs  a  way  to  retain  corporate  knowledge  and  benchmark  itself  against 
others. 

For  example,  at  NWC  the  number  of  requests  for  technical  publication 
and  technical  memorandum  report  numbers  is  much  higher  than  the 
number  of  reports  actually  submitted  to  the  Technical  Information 
Department  for  inclusion  in  retrievable  databases.  The  difference 
represents  new  technology  developed  at  DOD  expense  that  will  probably  be 
duplicated  because  a  literature  search  will  not  show  its  existence. 


Procedures  at  Micro-Level 

Micro-level  procedures  could  include  written  documentation, 
presentations,  and  training  sessions.  Keeping  daily  logs  and  notebooks  is 
important. 

Processes  at  Macro-Level 

Macro-level  processes  such  as  written  documentation,  presentations, 
and  training  sessions  help  share  information. 

Corporate  Databases 

Corporate  databases  could  be  constructed  that  are  categorized  by 
specific  topic.  Scientists  and  engineers  could  search  to  determine  what 
ideas  have  been  proposed,  how  much  development  has  occurred,  and  what 
processes  or  ideas  resulted  in  failures,  or  which  ideas  are  on  the  back 
burner  because  they  are  currently  considered  too  advanced.  Databases 
could  also  be  used  to  construct  "what  if  scenarios  with  all  of  the  available 
information. 
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CREATIVITY 


Disposition  of  Ideas 

One  element  of  an  R&D  institution  ’s  a  process  oi  an  advisory  pool  for 
identifying  types  of  ideas  and  delegating  responsibility  for  routing  each 
idea  to  its  best  disposition.  For  example,  part  of  the  process  could  require  a 
search  to  determine  previous  history  of  similar  concerns,  or  other 
important  information.  Components  of  any  idea  system  should  include 
well-thought-out  questions,  good  analysis  (including  trade-off  studies),  and 
good  documentation.  People  who  submit  ideas  and  proposals  for  research 
projects  need  the  resources  available  to  help  them  understand  and  more 
clearly  define  what  they  are  trying  to  say.  Three  basic  types  of  ideas  at 
NWC  are 

Ideas  That  End  Up  As  Funding  Requests  for  Research.  An  organization 
needs  a  scientific  or  logical  system  for  determining  which  research  projects 
best  fit  the  overall  mission  and  customer  needs.  A  research  organization 
should  have  a  prioritizing  process  for  assessing  availability  of  and  lead 
time  for  possible  acquisition  of  resources  for  proposed  research  projects. 
This  process  serves  as  a  decision  aid  for  determining  which  projects  to 
pursue.  Additionally,  a  process  should  be  in  place  for  allocating  people, 
tools,  equipment,  and  funds  among  existing  research  projects.  Both 
processes  need  to  be  impersonal. 

Ideas  That  Need  A  Process-Improvement  Team.  Experts  could  be 
available,  either  individually  or  in  small  groups,  to  address  important 
issues  on  short  notice. 

Ideas  That  are  Candidates  for  the  Suggestion  System.  This  type  does 
not  need  a  process-improvement  team.  These  ideas  are  the  "just  do  it" 
kind. 


Participation  in  R&D  Consortia 

A  trade-off  study  of  the  advantages  and  disadvantages  of  belonging  to 
an  R&D  consortium  could  be  enlightening.  Among  the  factors  that  could  be 
addressed  are  types  of  R&D  consortia,  advantages  and  disadvantages  of 
participation,  relative  effectiveness  of  the  existing  groups,  and  the 
possibility  of  forming  one  that  is  specific  to  NWC  needs. 
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MISSION  D1 


NWC  PHILOSOPHY  STATEMENT 


ANNUAL 

EXTERNAL  QUALITY 
MEASURES 


•  Quality  Is  First  Among  Equals 

We  Will  commit  to  quality  products  to  the  Fleet  by  understanding  our 
customers  expectations,  providing  products  and  services  that  conform  to 
performance,  cost,  and  schedule  rectw.>i:<»enta.  artuConiinudlly  improving  ^ui 
processes  to  ensure  the  most  effective  internal  operations 

•  People  are  Important 

The  most  important  asse*  of  the  Naval  Weapons  Center  is  our  people  and  the 
military,  civilian,  and  contractor  team.  We  will  invest  in  training  and 
development  of  our  people  throughout  their  careers,  provide  a  stimulating 
environment;  provide  opportunity  for  growth,  seff-management  and  utilization 
of  their  talents,  improve  the  quality  of  tneir  work  life,  and  encourage  mutual 
respect,  teamwork,  and  a  sense  of  community 

•  Individual  Coi  tnbution 

Each  and  every  employee  has  the  responsibility  arKf  obligation  to  actively 
panicipate  m  and  influence  the  future  of  the  Naval  Weapons  Center 
Management  has  the  responsibility  and  obligation  to  listen,  enable,  and 
implement. 

•  Work  Ethic 

Individual  and  team  excellence,  honesty,  credibility,  accountability,  and 
dedication  are  highly  valued  at  the  Naval  Weapons  Center.  These  traits 
ensure  the  integrity  of  our  decisions  and  technical  advice  Being  a  ‘China 
Laker'  means  a  ‘can-do*  attitude  and  ‘going  the  extra  mile*  to  get  the  job 
done 

•  Technical  and  Management  innovation 

The  willirtgness  to  try  new  approaches  and  apply  unique  solutions  is  a  vital 
element  of  the  Naval  Weapons  Centers  success.  Creativity,  innovation  and 
responsible  nsk  taking  are  encouraged,  nurtured,  and  rewarded  to  give  rise  to 
better  and  lower-cost  weapon  systems  We  will  continue  to  emphasize 
additional  management  innovations  that  provide  our  personnel  with  the 
freedom,  facilities,  resources,  remuneration,  and  ffexibifity  that  are  conducive 
to  creativity 

•  Open  and  Effective  Communication 

Effective  communication  through  all  levels  of  the  organization  is  absolutely 
vital  to  the  success  of  the  Naval  Weapons  Center. 

•  Full-Spectrum  Support 

Our  expertise  covers  the  full  range  of  the  weapons  development  process  from 
basic  and  applied  research  to  prototype  hardware  fabncation.  production 
support,  and  Fleet  support  While  we  concentrate  our  efforts  in  enhancing  the 
tei^nology  base  and  m  developing  and  integrating  new  systems,  we  will 
maintain  a  full  balance  of  capability  that  will  ke^  us  dearly  in  touch  not  only 
with  advancing  technology  but  also  with  the  probi^s  and  needs  of  the  Fleet 


The  Centers  mission  is  lo  De  !r>e  onocipai  Nav  > 
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NWC  DEFINI 


We  carry  out  our  vision  of  what  we  would  like  our  crcc 
mission  elements 

•  Research,  design,  development,  test,  and  systems 
systems  into  strike,  ASUW.  and  AAW  aircraft 

•  Research,  design,  development  and  test  of  tactical  r 

•  Operation,  maintenance,  and  improvement  of  a  natic 
and  evaluation 

•  Research,  design,  development,  and  test  of  paracnu’ 

•  Studies  of  naval  warfare  systems  for  strike  ASUW  - 

•  Promotion  and  maintenance  of  fundamental  research 
areas 

•  Operation,  maintenance,  and  improvement  of  a  nava 
RDT&E.  range  users,  tenant  activities,  and  surrounc 


NWC  QUA 


Achieve  customer  satis! 


MISSION  DEPLOYMENT 


NWC  MISSION 

OQSSiOO  IS  10  M:  tho  Qoncipai  Navy  Rosaarch.  OovelOQfneni  Test,  and  EvaiuatK>n  Center 
aitare  systems  lexcept  antisubmanne  warfare  systems)  and  missile  weapon  systems 
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NWC  DEFINITIVE  MISSION 


of  what  we  would  like  our  organization  to  be  through  a  dedication  to  the  followmg 

velopnnent.  test.  ar>d  systems  integration  of  air  weapons  and  associated  aircraft  weapon 
SUW.  and  AAW  aircraft. 

vetopment.  and  test  of  tactical  missiles  for  any  naval  platform 

ice.  and  improvement  of  a  national  test  range  tor  weapons  and  weapon  systems  testing 

yelopmeni.  and  test  of  parachute  systems 

tire  systems  for  stnke.  ASUW.  AAW  and  other  warfare  areas 

^nce  of  fundamental  research  and  the  technology  base  to  support  the  above  mission 

ice.  and  improvement  of  a  naval  base  and  air  faoiity.  including  SAR.  to  support 
i  tef>ant  acbvtties.  and  surrouryjing  communities 


NEXT  ECHELON  APPROVED 
LEADERSHIP  AREAS 

NWC  IS  offioaily  responsible  for  Navy-wide  leadership  m 

•  Air  warfare  analysis  (with  NADC)  (antiair.  anttsurlace.  and  strike  warfarei 

•  Air  combat  systems  and  integration  (with  NADC) 

•  Missiles  and  missile  subsystems 

•  Aircraft-launched  free-fall  weapons 

•  Aircraft  electronic  warfare 

•  Range  development  and  operation  (atr-to-air.  air-to-surface.  and 
surface-to-air  weapons,  air  electronic  warfare  and  parachute  systems) 

•  Explosives  (principally  scale-up) 

•  Missile  and  free-fall  weapon  fuzing  and  warheads 

•  Aenai  targets  (full-scale) 

•  Aerodynamic  deceleration  (parachute)  systems  and  components 

•  Aircraft  and  missile  nonnuclear  survivability  and  vulnerability 


NWC  QUALITY  POLICY 


chieve  customer  satisfaction  in  these  categories. 
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MITTED  TO  EXCELLENCE  THROUGH  TEAMWORK  AND  INNOVATION 
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INTERNAL  QUALITY 
MEASURES 


NEXT  ECHELON  APPROVED 
LEADERSHIP  AREAS 

NWC  IS  offioalty  responsible  for  Navy-wide  leadership  m 

•  warfare  analysts  {with  NADC)  (anftair.  antisudace.  and  stnKe  warfare) 

•  Air  combat  systems  arxj  integrabon  (with  NAOC) 

•  Missiles  and  missile  subsystems 

•  Aircraft-launched  freeman  weapons 

•  Aircraft  electronic  warfare 

•  Range  development  arxl  operation  (air-to-air.  atr-to-surface.  and 
surface-to-air  weapons:  air  electronic  warfare,  and  parachute  systems) 

•  Explosives  (pnnctpaily  scale-up) 

•  Mtssrie  and  free-fall  weapon  fuzing  and  warheads 

•  Aerial  targets  (fuH-scaie) 

■  Aerodynamic  deceleration  (parachute)  systems  and  components 

•  Aircraft  and  missile  nonnuclear  survivability  and  vulnerability 


ANNUAL  INTERNAL-CUSTOMER  SURVEY 
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